The present invention relates to data processing systems, and more specifically to large-scale, long-running data transfer to data storage systems.
In cloud computing, files often are distributed from a producer to one or more geographically distributed consumers. Among the simplest examples is a single file needed by an application such as a configuration file, a whitelist/blacklist file, a data dictionary, or a mapping file that needs to be delivered to a handful of web servers. At the complex end of the spectrum, there could be an entire set of search indices that need to be distributed to serving clusters in multiple data centers.
Improved mechanisms for efficiently and reliably distributing files within a cloud network or any other distributed storage network would be beneficial.